A lighting plan generator

ABSTRACT

A lighting plan generator configured to generate a lighting plan comprising a map of locations of a plurality of light units located on a ceiling of an interior space, the lighting plan generator being configured to generate the lighting plan from overlapping images captured by a mobile device from different locations as the mobile device moves through the interior space. The lighting plan generator comprises: an input configured to receive at least two images, the at least two images comprising at least partially overlapping areas; an image rectifier ( 160 ) configured to rectify the at least two images; an image data combiner ( 406 ) configured to combine data from the rectified at least two images, wherein a lighting plan ( 413, 415 ) is generated from the combined data from the rectified at least two images.

TECHNICAL FIELD

The present disclosure relates to a lighting plan generator system andmethod for generation of a lighting plan based on combining images orimage data. Particularly the present disclosure relates to a lightingplan generator system and method for generating a lighting plan based oncombining the image data either by blending images or extracted imagedata. The present disclosure relates to the use of such generatedlighting plans for inspection purposes or commissioning purposes.

BACKGROUND

Generating a lighting plan for a large area installation is a usefuloperation for many reasons.

Monitoring or managing a large (retail) area filled with many lightunits (for example 500 to 1500 light units within a 50 m² area) benefitsfrom an accurately determined lighting plan. For example forcommissioning purposes the lighting plan may be checked against theinstallation plan to determine whether all of the light units have beencorrectly installed. Similarly for inspection purposes the lighting planmay be compared against a previously determined lighting plan to checkwhether there has been a change in the performance of the light unitsand thus identify where light units should be replaced or repaired.

Furthermore the accurately determined lighting plan may have other usessuch as enabling an indoor positioning system. The indoor positioningsystem may allow users to identify their location relative to a ‘known’position of the light units determined from the lighting plan. Theindoor positioning system may for ‘internal space’ applications providesuperior performance over conventional radio frequency positioningsystems such as satellite (GPS) and/or cellular positioning systems.

Although an initial installation plan may provide an expected lightingplan, this initial lighting plan may comprise errors. For example lightunits may be installed at different locations to the originally plannedlocation (for example because of some structural problem in thebuilding). Similarly some light units may be incorrectly installed orswapped with other light units during the lighting system installationprocess.

The initial lighting plan may therefore not be suitable for the task itis to be used in. For example in an indoor positioning system thelighting plan should have the position of the light units to within 10to 20 cm in the X, Y and Z dimensions.

Such errors therefore require a post installation lighting plan to bedetermined. Conventionally this is generated manually and often by usinga paper version of the installation plan as a template, a device toidentify the light (when located underneath the light unit) and a pen ormarker to mark up any changes on the template. This is both timeconsuming, costly, inflexible and is also prone to mistakes.

For example once a lighting system has been commissioned the generatedlight map may be checked or inspected on a regular basis to determinewhether the light units are still working and/or in the correctposition. The inspection of installed light units and the generation ofa report including the brand, quantity, status, energy usage of thelighting system based on the inspection would be similarly both timeconsuming and prone to mistakes.

SUMMARY

The following provides a technique and apparatus for providing (andenabling the inspection of) a lighting system plan or lighting plan. Thetechnique and apparatus is configured to generate (or receive) multipleoverlapping images of the environment from various locations. Theseoverlapping images may be rectified and then data from the imagescombined. The combination of data from the images in some embodiments isachieved by combining the rectified images directly to form a blendedimage and then determining light units from the blended image. Thecombination of data from the overlapping images in some otherembodiments is performed by initially determining light unit data(otherwise known as a sparse representation of the image) from each ofthe overlapping images and then combining this light unit data.

Thus in some embodiments the concept may be implemented within a deviceor apparatus with a camera configured to capture the overlapping images.In some further embodiments the device or apparatus may be further inconnection with a processing system configured to manage and process theoverlapping image data.

The processing system may thus in some embodiments be configured torectify the overlapping (ceiling) images taken by the camera, stitch ormerge the rectified images into a single image, detect or determinelighting points from the single image and classify the type and statusof images. From this information the lighting plan can be generated orcompared against a previously determined lighting plan in order toeither generate a lighting plan or verify the lighting plan or inspectthe lighting system. The processing system may furthermore in some otherembodiments be configured to analyze each image to determine light unitsin each image and classify their type and status. Furthermore therelative location of each of these determined light units in each imagecan be stored. Using the relative positions of the light units frommultiple images a single combined map of light unit positions can bedetermined.

According to one aspect disclosed herein, there is provided a lightingplan generator configured to generate a lighting plan from overlappingimages, the lighting plan comprising a map of locations of a pluralityof light units located on a ceiling of an interior space, and thelighting plan generator comprising: an input configured to receive atleast two images from a mobile device, the at least two imagescomprising at least partially overlapping areas captured by the mobiledevice from different locations as the mobile device moves through saidinterior space; an image rectifier configured to rectify the at leasttwo images; an image data combiner configured to combine data from therectified at least two images, wherein the lighting plan is generatedfrom the combined data from the rectified at least two images.

The image data combiner may comprise an image combiner configured togenerate a merged image from the rectified at least two images, whereinthe lighting plan generator may comprise a light unit identifierconfigured to analyze the merged image to generate the lighting plan.

In such a manner the generation of the lighting plan may be performedbased on receiving image data in the form of bitmap or other image dataformats, processing the images to produce images with common featuresand then stitching or combining these images together based on thesecommon features identified in the images in order that the combinedimage may then be analyzed and used to produce a lighting plan.

The lighting plan generator may comprise a light unit identifierconfigured to analyze the at least two images individually to determineand identify any light units within the image, wherein the image datacombiner may be configured to combine the determined and identifiedlight unit information within the individual images to generate thelighting plan.

In such a manner the generation of the lighting plan may be performed byanalyzing each image individually and identifying the light units withineach image. The data of the identified light units may then be combinedto produce the lighting plan. In these embodiments the complete bitmapneed not be stored, or transmitted from a capture device to a processingor analyzer. In other words, only the determined light unit dataassociated with each image is stored or transmitted and thus the amountof data stored or transmitted may be reduced and/or the captured imagequality (or image pixel density) may be increased. Furthermore byanalyzing the image individually the later generation of the lightingplan by combining the image data may be performed using fewer processingand memory resources.

The light unit identifier may be configured to identify any light unitswithin the image from an identifier based on a modulation of visiblelight of the light unit or based on an additional infrared source in orwith the light unit and modulation of the light emitted by the infraredsource.

In such a manner the light units may be identified using the identifierfrom the light unit modulation or based on modulation from anyassociated infrared source associated with the light unit.

A lighting plan generating system may comprise: a device configured tocapture the at least two images, the at least two images comprising atleast partially overlapping areas; and the lighting plan generator asdiscussed herein, wherein the device may be configured to transmit theat least two images to the lighting plan generator.

In such embodiments the device configured to capture the at least twoimages may be a separate device or apparatus from the device configuredto generate the lighting plan. As such the device for capturing theimages may be any suitable device or apparatus for capturing images suchas mobile phone, a robot device, a drone or similar and as such may bechosen to suit the environment within which the lighting system has beeninstalled.

The device may be controlled by the lighting plan generator.

In such embodiments the lighting plan generator may control the devicein order to capture suitable overlapping images from which the lightingplan may be generated.

The device may be an autonomous device.

In such embodiments the device configured to capture the overlappingimages may be configured to operate with minimal or no user input andthus not require costly and potentially error introducing manual input.

A lighting commissioning system may comprise a lighting system generatoras discussed herein, the lighting commissioning system may furthercomprise a report generator configured to generate a commissioningreport based on the determined lighting plan.

In such embodiments the lighting commissioning system may be able togenerate suitable outputs for enabling the commissioning of lightingsystems in an efficient manner and with minimal errors.

A lighting commissioning system may comprise a lighting system generatoras discussed herein, the lighting commissioning system may furthercomprise: a lighting plan determiner configured to determine aninstallation lighting plan; a lighting plan comparator configured tocompare the installation lighting plan with the determined lightingplan; and a report generator configured to generate a commissioningreport based on the comparison of the installation lighting plan withthe determined lighting plan.

In such embodiments the lighting commissioning system may be able tocompare the generated lighting plan against the known installationlighting plan in order to determine whether the lighting system has beencorrectly installed or where there is an incorrect installation thedifferences which may be used by any later lighting system controller toallow for the error.

When no installation plan is available, the determined lighting plan maybe stored and made available for retrieval at a later date.

A lighting inspection system may comprise a lighting system generator asdiscussed herein, the lighting inspection system may further comprise: alighting plan determiner configured to determine a predeterminedlighting plan; a lighting plan comparator configured to compare thepredetermined lighting plan with the determined lighting plan; and areport generator configured to generate an inspection report based onthe comparison of the predetermined lighting plan with the determinedlighting plan.

According to a second aspect there is provided a computer programproduct comprising code embodied on one or more computer-readablestorage media and/or being downloadable therefrom, and being configuredso as when run on a lighting plan generator configured to generate alighting plan from overlapping images, the lighting plan comprising amap of locations of a plurality of light units located on a ceiling ofan interior space, and the lighting plan generator being configured toperform operations of: receive at least two images from a mobile device,the at least two images comprising at least partially overlapping areas;rectify the at least two images captured by the mobile device fromdifferent locations as the mobile device moves through said interiorspace; combine data from the rectified at least two images; and generatethe lighting plan from the combined data from the rectified at least twoimages.

The operation of combining data from the rectified images may comprisethe operation of generating a merged image from the rectified at leasttwo images, and the operation of generating the lighting plan from thecombined data may comprise the operation of analyzing the merged imageto generate the lighting plan.

The lighting plan generator may further be configured to perform theoperation of analyzing the at least two images individually to determineand identify any light units within the image, and wherein the operationof combining data from the rectified images may comprise performing theoperation of combining the determined and identified light unitinformation within the individual images, and the operation ofgenerating the lighting plan from the combined data may compriseperforming the operation of generating the lighting plan from thedetermined and identified light unit information.

The operation of analyzing the at least two images individually todetermine and identify any light units within the image may compriseperforming the operation of identifying any light units within the imagefrom an identifier based on a modulation of visible light of the lightunit or based on an additional infrared source in or with the light unitand modulation of the light emitted by the infrared source.

The lighting plan generator may further be configured to perform theoperation of capturing the at least two images, the at least two imagescomprising at least partially overlapping areas.

The lighting plan generator may further be configured to perform theoperation of controlling the capturing of the at least two images.

A lighting commissioning system may comprise the computer programproduct comprising code embodied on one or more computer-readablestorage media and/or being downloadable therefrom, and being configuredso as when run on a lighting plan generator configured to generate alighting plan from overlapping images and further be configured toperform the operation of generating a commissioning report based on thedetermined lighting plan.

A commissioning system may comprise the computer program productcomprising code embodied on one or more computer-readable storage mediaand/or being downloadable therefrom, and being configured so as when runon a lighting plan generator configured to generate a lighting plan fromoverlapping images and further be configured to perform the operationsof: determining an installation lighting plan; comparing theinstallation lighting plan with the determined lighting plan; andgenerating a commissioning report based on the comparison of theinstallation lighting plan with the determined lighting plan.

A lighting inspection system may comprise the computer program productcomprising code embodied on one or more computer-readable storage mediaand/or being downloadable therefrom, and being configured so as when runon a lighting plan generator configured to generate a lighting plan fromoverlapping images and further be configured to perform the operationsof: determining a predetermined lighting plan; comparing thepredetermined lighting plan with the determined lighting plan; andgenerating an inspection report based on the comparison of thepredetermined lighting plan with the determined lighting plan.

According to a third aspect there is provided a method of generating alighting plan from overlapping images, the lighting plan comprising amap of locations of a plurality of light units located on a ceiling ofan interior space, the method comprising: receiving at least two imagesfrom a mobile device, the at least two images comprising at leastpartially overlapping areas captured by the mobile device from differentlocations as the mobile device moves though said interior space;rectifying the at least two images; combining data from the rectified atleast two images; and generating the lighting plan from the combineddata from the rectified at least two images.

Combining data from the rectified images may comprise generating amerged image from the rectified at least two images, and generating thelighting plan from the combined data may comprise analyzing the mergedimage to generate the lighting plan.

The method may further comprise analyzing the at least two imagesindividually to determine and identify any light units within the image,and wherein combining data from the rectified images may comprisecombining the determined and identified light unit information withinthe individual images, and generating the lighting plan from thecombined data comprises generating the lighting plan from the determinedand identified light unit information.

Analyzing the at least two images individually to determine and identifyany light units within the image may comprise identifying any lightunits within the image from an identifier based on a modulation ofvisible light of the light unit or based on an additional infraredsource in or with the light unit and modulation of the light emitted bythe infrared source.

Receiving the at least two images may comprise capturing the at leasttwo images, the at least two images comprising at least partiallyoverlapping areas.

The method may further comprise controlling the capturing of the atleast two images at an apparatus separate from the apparatus performingrectifying the at least two images.

A method for commissioning a lighting system comprising: generating alighting plan as discussed herein; and generating a commissioning reportbased on the determined lighting plan.

A method for commissioning a lighting system may comprise: generating alighting plan from overlapping images as described herein; determiningan installation lighting plan; comparing the installation lighting planwith the determined lighting plan; and generating a commissioning reportbased on the comparison of the installation lighting plan with thedetermined lighting plan.

A method for inspecting a lighting system may comprise: generating alighting plan from overlapping images as described herein; determining apredetermined lighting plan; comparing the predetermined lighting planwith the determined lighting plan; and generating an inspection reportbased on the comparison of the predetermined lighting plan with thedetermined lighting plan.

BRIEF DESCRIPTION OF THE DRAWINGS

To assist understanding of the present disclosure and to show howembodiments may be put into effect, reference will be made by way ofexample to the accompanying drawings in which:

FIG. 1 is a schematic block diagram of a lighting plan generating systemaccording to some embodiments,

FIG. 2 is a schematic block diagram of functional components of a firstlighting plan generator according to some embodiments,

FIG. 3 shows a flow diagram of a first lighting plan generation methodimplementing the lighting plan generator as shown in FIG. 2,

FIG. 4 is a schematic block diagram of functional components of a secondlighting plan generator according to some embodiments,

FIG. 5 shows a flow diagram of a second lighting plan generation methodimplementing the lighting plan generator as shown in FIG. 4,

FIG. 6 shows an example operation of a user device/autonomous device asshown in FIG. 1,

FIG. 7 shows an example sequence of overlapping images, light unit imagedata, and light unit data for the second lighting plan generationmethod,

FIGS. 8 and 9 show example combined light unit data before and afterapplying triangulation to the light unit image data,

FIG. 10 shows an example rectification operation with respect to anexample image,

FIG. 11 shows an example image rectification and combination methodaccording to the first lighting plan generation method,

FIG. 12 shows an example combination operation with respect to a pair ofexample images,

FIG. 13 shows an example light unit identifier operation with respect toan example image, and

FIG. 14 shows an example lighting plan overlaying an installation plan.

DETAILED DESCRIPTION OF EMBODIMENTS

The concept implemented in the embodiments described herein isconfigured to generate a lighting plan for an interior space withinwhich is located a lighting system comprising light units located on asurface (typically the ceiling) for providing illumination for theinterior space. The generated lighting plan may then be employed withina range of various applications such as, for example, lighting systemcommissioning, lighting system inspection, and lighting system locationestimation.

The lighting plan may comprise light unit location or positioninformation relative to a defined datum or data. The defined datum ordata may be a structural element of the interior space and the locationor position information is ‘absolute’ location or position information.In some embodiments the defined datum or data may be other light unitsand the location or position information is ‘relative’ location orposition information.

In some situations each light unit within the light plan may furthermorebe identified with a code or hidden code. Each light unit may have anembedded unique identifier code which can be determined and be used touniquely identify the light unit. The identifiable code can be embeddedin the light units such as light emitting diodes as well as halogen,fluorescent and high-intensity discharge lamps. The identifier may bebased on a modulation of visible light of the light unit or by placingan additional infrared source in or with the light unit and modulatingthe light emitted by this infrared source. LEDs are particularlywell-suited for coded light systems since they can be modulated at ahigh frequency allowing the modulated light emissions to be modulatedbeyond 100 Hz rendering the data modulation substantially imperceptibleto the human visual system.

The unique identifier or code emitted by the light unit may be utilizedby a wide variety of tools and applications, including theidentification of one or more specific light units in the presence ofnumerous light units, which in turn enables applications such as theindoor positioning scheme.

With respect to FIG. 1 example apparatus for generating a lighting planaccording to some embodiments are shown. In FIG. 1 the lighting plangenerating system comprises a mobile user device/autonomous device 100configured to capture overlapping images of the environment and acentral management system 199 configured to process the overlappingimages.

It is understood that in some embodiments the aspects of the centralmanagement system 199 may be implemented within the userdevice/autonomous device 100. In other words in some embodiments thefunctionality of the central management system 199 and the userdevice/autonomous device 199 may be implemented within a singleapparatus or device. Similarly, in some embodiments, aspects or parts ofthe processing operation described hereafter may be distributed betweenthe user device/autonomous device 100 and the central management system199.

The user device/autonomous device 100 may be implemented in a range ofapparatus. For example the user device/autonomous device functionalitymay be a conventional user equipment or mobile phone held by a user. Theuser equipment may be configured to capture the overlapping images ofthe light units as the user walks through the space or retail area alonga defined or random path. Similarly the user device may be implementedas a tablet, laptop or other computing device with a camera which isheld. The user device may be mounted on wheels, or a wheeled or trackedsupport (such as a trolley) and which can be pulled or pushed throughthe interior space capturing overlapping images of the ceiling (or othersurface on which the light units are mounted).

In some environments the user device/autonomous device 100 may beself-propelled or mounted on a self-propelled support. For example theuser device may be implemented on a wheeled trolley and propelled byelectrical motors.

The self-propelled device may furthermore may be remote controlled, forexample using a wireless controller. The self-propelled device may bemanually controlled, autonomous or semi-autonomously operated.

Furthermore in some embodiments the user device may be a flying deviceor drone. For example the flying user device/autonomous device may be aheavier than air device (such as an axially stabilized helicopter) or alighter than air device (such as an autonomous airship).

The user device/autonomous device 100 may comprise a camera or imagecapturing device 110. The camera 110 may be configured to capture imagesof the environment within which the user device is operating. Typicallythe camera is orientated or stabilized such that the images captured areoverlapping images of the ceiling of the interior space. However in someembodiments the camera 110 may be configured to capture images of anysurface on which the light units are employed. The camera 110 may be anysuitable camera or imaging array. In some embodiments the camera 110 maybe configured to further determine a depth or distance from the camerato the surface. For example in some embodiments the camera 110 is astereoscopic image determiner configured to determine a depth ordistance based on comparing the parallax differences.

The user device/autonomous device 100 may further comprise alocation/orientation detector 120. The location/orientation detector 120may be configured to provide some location and/or orientationinformation which may be used to tag the captured images and be used inthe processing of the overlapping images. The location/orientationdetector 120 may for example comprise a gyroscope or digital compass.

The user device/autonomous device 100 may further comprise a processor130 and memory 135. The processor 130 can in some embodiments beconfigured to execute various program codes. The implemented programcodes in some embodiments comprise image capture control, imageprocessing, and image encoding as described herein.

The user device/autonomous device 100 may further comprise memory 135.The implemented program codes can in some embodiments be stored forexample in the memory 135 for retrieval by the processor 130 wheneverneeded. The memory 135 could further provide a section for storing data,for example image data in accordance with the application as describedherein.

The user device/autonomous device may further comprise a transmitter 140(or a transceiver).

The user device/autonomous device 100 may thus be wirelessly incommunication with the central management system 199. The transmitter(or transceiver) may communicate with other apparatus by any suitableknown communications protocol. For example in some embodiments thetransmitter can use a suitable universal mobile telecommunicationssystem (UMTS) protocol, a wireless local area network (WLAN) protocolsuch as for example IEEE 802.X, or a suitable short-range radiofrequency communication protocol such as Bluetooth.

The user device/autonomous device 100 may further comprise a userinterface 150.

The user interface (UI) 150 may enable a user to input commands to theuser device 100. The user interface 150 may be a keypad or othersuitable input device.

The user device/autonomous device 100 may further comprise a display155. The display 155 is configured to present information about the userdevice/autonomous device. In some embodiments a touch screen may provideboth user interface 150 input and display 155 output functions.

In the following embodiments the user device/autonomous device 100 isconfigured to provide to the central management system 199 a series ofoverlapping images which cover the ceiling of the interior for which thelighting map is being generated. In such embodiments the userdevice/autonomous device 100 camera 110 is configured to capture theimages, which can be encoded using the processor 130 and passed to thetransmitter 140 for transmission to the central management system 199.In some embodiments the location detector 120 may be further configuredto pass location/orientation estimates to the processor and theselocation/orientation estimates may be attached to the images to also betransmitted to the central management system 199.

With respect to FIG. 6 an example user device 100 moving through aninterior space capturing overlapping images is shown. The user device100 is shown at various positions within the interior space, andrepresents a series of time instances where the user device 100 is movedin the direction shown by arrow 621 along the floor. At each instant thecamera 110 is configured to capture an image of which each imageoverlaps at least partially with one other image. Thus at the firstinstant the camera 110 is configured to capture an image comprising thefirst 601 light unit and the second light unit 603. At the secondinstant the camera 110 is configured to capture an image comprising thesecond light unit 603, the third light unit 605 and partially the fourthlight unit 607. At the third instant the camera 110 is configured tocapture an image comprising the fourth 607 light unit and the fifthlight unit 609. At the fourth instant the camera 110 is configured tocapture an image comprising the fifth 609 light unit and the sixth lightunit 611.

The central management system 199 may comprise a receiver 160 (ortransceiver). The user central management system 199 may thus bewirelessly in communication with the user device 100. For example thecentral management system 199 may be configured to receive theoverlapping images captured by the user device 100. The receiver (ortransceiver) may communicate with other apparatus by any suitable knowncommunications protocol. For example in some embodiments the receivercan use a suitable universal mobile telecommunications system (UMTS)protocol, a wireless local area network (WLAN) protocol such as forexample IEEE 802.X, or a suitable short-range radio frequencycommunication protocol such as Bluetooth.

The central management system 199 may further comprise a processor at170 and memory 180. The processor 170 can in some embodiments beconfigured to execute various program codes. The implemented programcodes in some embodiments comprise image processing, lighting mapgeneration, commissioning report generation using the generated lightingmap, and inspection report generation using the generated lighting map.

The central management system may further comprising memory 180. Theimplemented program codes can in some embodiments be stored for examplein the memory 180 for retrieval by the processor 170 whenever needed.The memory 180 could further provide a section for storing data, forexample image data in accordance with the application as describedherein.

The central management system 199 a further comprise a user interface190.

The user interface (UI) 190 may enable a user to input commands to thecentral management system 199. The user interface 190 may be a keypad orother suitable input device.

The central management system may further comprise a display 195. Thedisplay 195 is configured to present information about the centralmanagement system. In some embodiments a touch screen may provide bothuser interface 190 input and display 195 output functions.

With respect to FIG. 2 functional entities implemented within thecentral management system 199 for a first lighting plan generator areshown in further detail. In this example the lighting plan generator isemployed within a lighting system inspection system and as such comparesthe light unit information within the lighting plan with a predetermined(where it is available) lighting plan. The predetermined lighting plandescribed with respect to this example may be a previously determinedlighting plan or may be an installation lighting plan.

In some embodiments of the central management system 199 furthercomprises an image rectifier 201. The image rectifier 201 may beconfigured to receive the overlapping images and process the image tocompensate for camera lens and/or user device orientation errors suchthat data within the images may be combined. The image rectifier 201 mayfor example be configured to identify the shooting angle or the poseangle of the camera. Following the determination of the shooting angleof the image the image rectifier may then be configured to process theimage based on the shooting angle to form a rectified image. Therectified image may then be passed to the image combiner 203.

In some embodiments the central management system 199 further comprisesan image combiner 203. The image combiner 203 may be configured toreceive the rectified images from the image rectifier 201 and combine orblend the images to form a single combined or blended image. Thiscombined or blended image may then be passed to a light unit identifier205.

In some embodiments the central management system 199 comprises a lightunit identifier 205. The light unit identifier 205 may be configured toreceive the blended or combined image from the image combiner 203.Furthermore the light unit identifier 205 may be configured to identifyfrom the blended image the light units within the image. Furthermore thelight unit identifier 205 may be configured to identify the type oflight unit, the position (or relative position) of the light unit, andthe status of the light units. The light unit identifier 205 may also beconfigured to determine other suitable parameters or characteristics ofthe light units identified. For example the light unit identifier maydetermine the make of the light unit. The identified light unit valuesfrom the blended image can then be passed to a lighting comparator 213.

In some embodiments the central management system 199 further comprisesa lighting plan determiner 211. The lighting plan determiner 211 may beconfigured to determine and/or retrieve a previously determined lightingplan. The previously determined lighting plan may then be passed to thelighting comparator 213. The lighting plan determiner 211 may beconfigured to output a light plan in the same format as the informationgenerated by the light unit identifier 205.

In some embodiments the central management system 199 comprises alighting plan comparator 213. The lighting plan comparator 213 may beconfigured to compare the information from the light unit determineragainst the information from the lighting plan determiner. The output ofthe lighting plan comparator 213 can then be passed to a reportgenerator 221.

In some embodiments the central management system 199 comprises a reportgenerator 221. The report generator may be configured to receive theoutput of the lighting plan comparator 213 and further be configured togenerate a report based on the differences between the lighting planinformation from the lighting plan determiner and the identified lightunit information.

In some embodiments, where there is no predetermined lighting planavailable from the lighting plan determiner 211, the lighting plancomparator 213 may issue an indicator to the report generator 221 tooutput the identified light unit information as a new lighting plan.This ‘new’ lighting plan may be stored and be available for retrieval ata later date by the lighting plan determiner 211.

With respect to FIG. 3 an example flow diagram of the operation of thecentral management system 199 as shown in FIG. 2 is shown.

In some embodiments the central management system 199 is configured toreceive the overlapping images from the user device. The overlappingimages may be received on an ongoing or sequential basis wherein theimages are uploaded to the central management system 199 as the userdevice captures images. In some embodiments the overlapping images maybe received as a batch process, wherein the user device 100 isconfigured to upload multiple images or all of the images at the sametime.

The operation of receiving the overlapping images is shown in FIG. 3 bystep 301.

The overlapping images may then be rectified by the image rectifier 201.

With respect to FIG. 10 an example warping of an image is shown whereinthe initial image 1001 is warped or rectified according to the methodsas described herein to form a warped (or mapped or rectified) image1003. In such an example the original image shows the ‘square’ featureas a non-regular quadrangle which when warped is shown as a ‘square’.

The operation of rectifying the overlapping images is shown in FIG. 3 bystep 303.

The rectified images can then be combined or blended to form a singlecombined or blended image by the image combiner 203.

With respect to FIG. 12 an example blending or combining of the imagesis shown wherein a first image 1201 and a second image 1203 are shown asbeing blended or combined to form a blended or combined image 1205.

The operation of generating a blended image from the rectified images isshown in FIG. 3 by step 305.

The combined or blended image may then be analyzed within the light unitidentifier to identify light units. The light units can be identified interms of the type of light unit, make of light unit, position of lightunit and status of the light unit.

With respect to FIG. 13 an example light unit identification operationis shown. Within the image the lighting units 1301 and 1303 areidentified as having 3 separate light source elements each.

The operation of identifying light units from the blended image is shownin FIG. 3 by step 307.

Furthermore in some embodiments it can be determined whether there is apredetermined lighting plan stored. This predetermined lighting plan canbe retrieved by the lighting plan determiner 211 and the predeterminedlighting plan may be passed to a lighting plan comparator 213.

The operation of determining a predetermined lighting plan is shown inFIG. 3 by step 309.

The identified light unit information may then be compared against theinformation from the predetermined lighting plan information.

The operation of comparing the information from the predeterminedlighting plan against information about the identified light units isshown in FIG. 3 by step 311.

Then lighting plan report may be generated, for example by the reportgenerator, based on the comparison between the information from thepredetermined lighting plan against the information about the identifiedlight units.

The operation of generating the lighting plan report based on thecomparison is shown in FIG. 3 by step 313.

Thus for example with respect to FIG. 14 is shown an example lightingplan report wherein the light units 1403, 1405 and 1407 are identifiedand located over the plan 1401 of the interior space.

With respect to FIG. 4 functional entities implemented within thecentral management system 199 for a further lighting plan generator areshown in further detail. The further lighting plan generator in thisexample is one where the lighting plan is generated from combining datadetermined from images rather than combining the bitmaps associated withthe images. In such embodiments as described herein as the image data isnot bitmaps data but determined data comprising coordinates and codesfrom the found and identified coded lights in the image. Thus in suchembodiments for a large area there is a significantly smaller amount ofstorage required or otherwise higher detail images can be ‘stored’enabling a higher accuracy of lighting unit determination. Furthermoreby applying the operation of finding and identifying the light unit fromeach image and then combining the image data in the form of theco-ordinates for the found and identified light units the combination or‘stitching’ of this data to form the lighting plan is much easier andrequires less processing capacity. Although in this example theprocessing of the image to determine the image data is performed withinthe central management system 199 it is understood that in someembodiments the operation of processing the image to determine the imagedata and/or the operation of stitching or combining the determined datamay be performed within the capture device.

Furthermore the examples as discussed hereafter enable an integrallighting plan comprising the light units and their codes (which wouldhave to be determined separately from the integral lighting plan methoddiscussed above).

In the following example the further lighting plan generator isimplemented within a lighting system commissioning system wherein thedetermined light unit information is compared against the installationlighting plan to determine whether the light units have been installedcorrectly. The overlapping images may be received on an ongoing orsequential basis wherein the images are uploaded to the centralmanagement system 199 as the user device captures images. In someembodiments the overlapping images may be received as a batch process,wherein the user device 100 is configured to upload multiple images orall of the images at the same time.

In some embodiments of the central management system 199 furthercomprises an image rectifier 401. The image rectifier 401 may beconfigured to receive the overlapping images and process each image tocompensate for camera lens and/or user device orientation errors suchthat data within the images may be combined. The image rectifier 401 mayfor example be configured to identify the shooting angle or the poseangle of the camera. Following the determination of the shooting angleof the image the image rectifier may then be configured to process theimage based on the shooting angle to form a rectified image. Therectified image may then be passed to a light unit identifier 405. Theimage rectifier 401 may furthermore scale the image such that each imageis uniformly scaled. For example the output of the image rectifier maybe such that each image each image the images may be analyzed and theresultant analysis output directly compared.

The camera lens errors or other non-linear features (and thecompensation) may be determined by comparing a sequence of images andusing identified features within the images comparing distances orangles between the features. For example any distances or angles betweenfeatures (such as identified light units) which differ between images(which have been rectified and scaled to compensate for the differencesin camera orientation) may be registered and used to generate a mappingor scaling to compensate for the differences. In other words theposition of the features (such as the light units) may be triangulatedfrom the images, with the camera lens errors or other non-linearfeatures appearing as differences between the triangulated positions andthe individual image position.

In some embodiments the central management system 199 comprises a lightunit identifier 405. The light unit identifier 405 may be configured toreceive the rectified images from the image rectifier 401. Furthermorethe light unit identifier 405 may be configured to identify from therectified image the light units within the image. This may be performedby determining and then identifying an associated code with the lightunit as discussed earlier. Furthermore the light unit identifier 405 maybe configured to identify the type of light unit, the position (orrelative position) of the light unit within the image, and the status ofthe light units. The light unit identifier 405 may also be configured todetermine other suitable parameters or characteristics of the lightunits identified. For example the light unit identifier may determinethe make of the light unit. The identified light unit values from theblended image can then be passed to a lighting data combiner 406.

FIG. 7 shows an example operation of the light unit identifier accordingto some embodiments.

Thus for example FIG. 7 shows a series of captured overlapping images701. The overlapping images comprise a first image 711, a second image713, a third image 715 and a fourth image 715. These images may, forexample, be captured using a camera 110 mounted on user device 100 suchas shown in FIG. 6 and as such the image capture position differs fromimage to image by a displacement in a single dimension.

These overlapping images may then be rectified (in other words mapped tocorrect any differences in scale, angle and lens distortion) and thenanalyzed to identify and light units within each of the images. Thisanalysis is shown in the series of image and light identifier labels703. Thus from the from the first image 711 is shown the identifiedlight units with the identifiers #32, #865 and #3241, from the secondimage 713 the light units with the identifiers #56, #435, #32, and #865are identified, from the third image 715 the light units with theidentifiers #56, #435, and #32 are identified and from the fourth image717 the light units with the identifiers #7451, #435, #56, and #32 areidentified. Furthermore the analysis of each image provides an imageposition which may then be associated with each light unit identifier.

In such a manner the generation of the lighting plan may be performed byanalyzing each rectified image individually and identifying the lightunits within each image. The identified light units within the image maybe registered with the light unit number or identifier and the positionof the light unit within the rectified image.

The data (the image identifier, the light unit identifier, and theposition within the rectified image) associated with the identifiedlight units may then be combined to produce the lighting plan. In theseembodiments the complete bitmap for the image need not be stored, ortransmitted from a capture device to a separate processing device oranalyzer.

In such embodiments only the determined light unit data associated witheach image is stored or transmitted and thus the amount of data storedor transmitted may be reduced and/or the captured image quality (orimage pixel density) may be increased. Furthermore by analyzing theimage individually the later generation of the lighting plan bycombining the image data may be performed using fewer processingresources and memory resources.

In other words the light unit identifier 405 is applied to individualrectified images to generate a ‘sparse’ image representation of theimages which are combined to form the lighting plan rather thangenerating a ‘complete’ image representation which is then analyzed bythe light unit identifier 205 such as shown in the earlier method.

In some embodiments the central management system 199 comprises alighting data combiner 406. The lighting data combiner 406 may beconfigured to receive the identified light unit information data andcombine this information in such a manner that the image data isconsistent between the overlapping ‘sparse’ images. The ‘combined’ lightunit information may then be passed to a lighting plan comparator 413.

In this example the images are captured and sent to the lighting plangenerator to be rectified, analyzed to determine the light units andpositions within each image and then the image data combined. In suchembodiments the capture device or the management system may comprise amodel of the used camera and the angle of the camera is also known. Therectification of the images may be performed (to correct for scale andangle and distortion) inside the device. The compensated coordinates andcodes from the light are stored. The combination of these ‘sparse’images to generate a complete lighting plan may be performed by a simpleoverlaying of the overlapping ‘sparse’ image values as all coordinatesare already compensated for.

However in some embodiments the rectification of the captured images isperformed by initially passing the unrectified images to a light unitidentifier, where the light units are identified and image coordinatesand codes identifying the specific light are stored for each identifiedlight. In such embodiments the camera may furthermore determine orestimate camera position and orientation (camera pose) values such asangle, height, and X-Y delta absolute for each captured image. This datamay be passed to a rectifier which stores or defines a model of thecamera and based on the camera model and camera pose values may map orconvert the co-ordinates in terms of scale, angle and lens-distortion tooutput ‘rectified’ co-ordinates which may be combined with other image‘rectified’ co-ordinates into a ‘full’ set of coordinates. Thecombination of these ‘rectified’ co-ordinates may be performed asdescribed hereafter.

Furthermore the capture device may further comprise the light unitidentifier and be configured to determine the light units and theirassociated rectified coordinates and codes from the images. Thisinformation may be stored on the capture device and uploaded to thelighting plan generator at a later time. Thus the lighting plancomparison and generation may be performed off-line as all of the imagecoordinates are already compensated for.

Another example may be to use the information on movement of the camerato determine the rectification mapping. In such examples by capturingmany pictures the lighting plan generator and/or capture device can‘learn’ or determine the camera movement/angle from the difference inthe light units within the images. Thus the system may determine anysuitable rectification or compensation mapping to correct the image.

With respect to FIGS. 8 and 9 the operation of performing imagecombination (or image data combination) before and after image (or imagedata) rectification/error compensation is shown. FIG. 8 for exampleshows example images 811, 813, 815, 817. In this example the images 811,813, 815 and 817 and the identified light units in each image are shown.Furthermore it can be seen that due to the differences in measurementlocations (due to errors in the image capturing process) and othercamera errors that the overlaying of the images produces when the imagesare combined light unit locations which are ‘blurred’ and do not have aconsistent point. This can be seen for example by the light units whichare shown as occurring in multiple locations represented by regions 801,803, 805 and 807.

However where the images (or the image data, such as the light unitcoordinates) are rectified or mapped to compensate for the errors in thecamera (such as lens distortion/aberration) and movement of the camerasuch that a single or consistent position for the light units can bedetermined such as shown in FIG. 9 wherein the images 811, 813, 815, and817 from FIG. 8 are rectified (or mapped) to form images 911, 913, 915and 917 respectively and which when combined have a single consistentlight unit position for each of the identified light units 901, 903,907, 905. In the example shown in FIG. 9 the rectification of the imageis represented by the ‘warping’ of the image boundary from a rectangleto an irregular shape.

In some embodiments the central management system 199 further comprisesa lighting plan determiner 411. The lighting plan determiner 411 may beconfigured to determine and/or retrieve an installation lighting plan.The installation lighting plan may then be passed to the lightingcomparator 413. The lighting plan determiner 411 may be configured tooutput the light unit information in the same format as the informationgenerated by the lighting data combiner 406.

In some embodiments the central management system 199 comprises alighting plan comparator 413. The lighting plan comparator 413 may beconfigured to compare the information from the lighting data combiner406 against the information from the lighting plan determiner 411. Theoutput of the lighting plan comparator 413 can then be passed to areport generator 415.

In some embodiments the central management system 199 comprises a reportgenerator 415. The report generator 415 may be configured to receive theoutput of the lighting plan comparator 413 and further be configured togenerate a report based on the differences between the lighting planinformation from the lighting plan determiner and the identified lightunit information.

With respect to FIG. 5 an example flow diagram of the operation of thecentral management system 199 as shown in FIG. 4 is shown.

In some embodiments the central management system 199 is configured toreceive the overlapping images from the user device. The overlappingimages may be received on an ongoing or sequential basis wherein theimages are uploaded to the central management system 199 as the userdevice captures images. In some embodiments the overlapping images maybe received as a batch process, wherein the user device 100 isconfigured to upload multiple images or all of the images at the sametime.

The operation of receiving the overlapping images is shown in FIG. 5 bystep 501.

The overlapping images may then be rectified by the image rectifier 401.

The operation of rectifying the overlapping images is shown in FIG. 5 bystep 503.

The rectified images can then be individually analyzed to determine thelight unit information by the light unit identifier 406. The light unitscan be identified in terms of the type of light unit, make of lightunit, position of light unit and status of the light unit.

The operation of identifying light unit information from the rectifiedimages is shown in FIG. 5 by step 505.

The identified light unit information may then be combined, to determinea light unit position from image light unit information from multipleimages.

The operation of determining the position of the light unit positionsfrom the overlapping image data is shown in FIG. 5 by step 507.

Furthermore in some embodiments the installation or predeterminedlighting plan is determined. This lighting plan can be retrieved by thelighting plan determiner 411 and the predetermined lighting plan may bepassed to a lighting plan comparator 413.

The operation of determining an installation or predetermined lightingplan is shown in FIG. 5 by step 509.

The identified light unit information may then be compared against theinformation from the installation lighting plan information.

The operation of comparing the information from the installationlighting plan against information about the identified light units isshown in FIG. 5 by step 511.

Then commissioning lighting plan report may be generated, for example bythe report generator, based on the comparison between the informationfrom the predetermined lighting plan against the information about theidentified light units.

The operation of generating the lighting plan report based on thecomparison is shown in FIG. 5 by step 513.

With respect to FIG. 11 an example rectification and combinationoperation such as performed by the image rectifier 201 and imagecombiner 203 is shown in further detail.

The rectification operation comprises a first stage of inputting theimages.

The operation of inputting images is shown in FIG. 11 by step 1103.

The first set of operations performed on the images may be a series ofimage registration operations. The registration operations aim todetermine a series of features which are common to at least two imagesand from which registration data may be determined in order to ‘rectify’or otherwise map the images and so enable the images to be combined.

With respect to the image registration operations the input images maybe resized in a medium resolution. From these images suitable featuresare found in the images. The determined or found features may be matchedover a series of images. Then images and matched feature subsets may beselected in order to generate registration data in order to rectify theimages and build the combined image.

The operation of registering the images is shown in FIG. 11 by step1105.

The registration data for determining the mapping of the images may thenbe generated. This may be performed for example by initially estimatingcamera parameters based on comparing the identified registrationfeatures from image to image for a subset of the features and images.These camera parameters or registration data may further be refinedusing more identified registration features or images.

The operation of generating the registration data for mapping the imagesis shown in FIG. 11 by step 1107.

The registration data may then be applied to the input images tocomposite the images in order to map the images (for rectificationpurposed) to form a single combined image.

The compositing of the images is shown in FIG. 11 by step 1109.

In some embodiments the compositing operation may correct fordifferences in any intrinsic and extrinsic camera error. For example insome embodiments the images may be corrected for exposure errors as wellas positional errors or lens aberration errors.

The composited image can then be output as a final image.

The operation of outputting the composited image is shown in FIG. 11 bystep 1111.

Other variations to the disclosed embodiments can be understood andeffected by those skilled in the art in practicing the claimedinvention, from a study of the drawings, the disclosure, and theappended claims. In the claims, the word “comprising” does not excludeother elements or steps, and the indefinite article “a” or “an” does notexclude a plurality. A single processor or other unit may fulfill thefunctions of several items recited in the claims. The mere fact thatcertain measures are recited in mutually different dependent claims doesnot indicate that a combination of these measures cannot be used toadvantage. A computer program may be stored and/or distributed on asuitable medium, such as an optical storage medium or a solid-statemedium supplied together with or as part of other hardware, but may alsobe distributed in other forms, such as via the Internet or other wiredor wireless telecommunication systems. Any reference signs in the claimsshould not be construed as limiting the scope.

1. (canceled)
 2. (canceled)
 3. (canceled)
 4. (canceled)
 5. (canceled)
 6. A lighting plan generating system comprising: a mobile device configured to capture the at least two images, the at least two images comprising at least partially overlapping areas captured by the mobile device from different locations as the mobile device moves through an interior space; and a lighting plan generator configured to generate a lighting plan from the overlapping images, the lighting plan comprising a map of locations of a plurality of light units located on a ceiling of an interior space, wherein the mobile device is configured to transmit the at least two images to the lighting plan generator; wherein the lighting plan generator comprises: an image rectifier configured to rectify the at least two images; an image data combiner configured to combine data from the rectified at least two images, wherein the lighting plan is generated from the combined data from the rectified at least two images.
 7. The lighting plan generating system as claimed in claim 6, wherein the mobile device is controlled by the lighting plan generator.
 8. The lighting plan generating system as claimed in claim 6, wherein the mobile device is an autonomous device.
 9. (canceled)
 10. (canceled)
 11. (canceled)
 12. (canceled)
 13. (canceled)
 14. (canceled)
 15. (canceled)
 16. A method of generating a lighting plan from overlapping images, the lighting plan comprising a map of locations of a plurality of light units located on a ceiling of an interior space, and the method comprising: receiving at least two images from a mobile device, the at least two images comprising at least partially overlapping areas captured by the mobile device from different locations as the mobile device moves through said interior space; rectifying the at least two images; combining data from the rectified at least two images; and generating the lighting plan from the combined data from the rectified at least two images.
 17. The method as claimed in claim 16, wherein combining data from the rectified images comprises generating a merged image from the rectified at least two images, and generating the lighting plan from the combined data comprises analyzing the merged image to generate the lighting plan.
 18. The method as claimed in claim 16, further comprising analyzing the at least two images individually to determine and identify any light units within the image, and wherein combining data from the rectified images comprises combining the determined and identified light unit information within the individual images, and generating the lighting plan from the combined data comprises generating the lighting plan from the determined and identified light unit information.
 19. The method as claimed in claim 18, wherein the determination and identification of the light units within the image further comprises classifying at least one of a type and a status of the respective light units.
 20. The method as claimed in claim 18, wherein the identification of the light units comprises identifying any light units within the image from an identifier based on a modulation of visible light of the light unit or based on an additional infrared source in or with the light unit and modulation of the light emitted by the infrared source.
 21. A lighting commissioning method comprising generating a lighting plan as claimed in claim 16, the lighting commissioning method further comprising: determining an installation lighting plan; comparing the installation lighting plan with the determined lighting plan; and generating a commissioning report based on the comparison of the installation lighting plan with the determined lighting plan.
 22. The lighting plan commissioning method as claimed in claim 21, configured so that when no installation plan is available, the determined lighting plan is stored and made available for retrieval at a later date.
 23. A lighting inspection method comprising generating a lighting plan as claimed in claim 16, the lighting inspection method further comprising: determining a predetermined lighting plan; comparing the predetermined lighting plan with the determined lighting plan; and generating an inspection report based on the comparison of the predetermined lighting plan with the determined lighting plan. 